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A  Fixed-Wing  Aircraft  Simulation  Tool  for  Improving  the 
Efficiency  of  DoD  Acquisition 

Scott  A.  Morton*  and  David  R.  McDaniel1 
DOD  HPCMP-CREATE™  Kestrel  Team,  Eglin  AFB,  FL  32542 

The  HPCMP  CREATE™-AV  Kestrel  software  product  is  a  modularized, 
multi-disciplinary  fixed-wing  virtual  aircraft  simulation  tool  incorporating 
aerodynamics,  structural  dynamics,  kinematics,  and  kinetics.  Kestrel  allows  cross¬ 
over  between  simulation  of  aerodynamics,  dynamic  stability  and  control, 
structures,  propulsion,  and  store  separation.  Kestrel  is  written  in  modular 
form  with  a  Python  infrastructure  to  allow  growth  to  additional  capabilities 
as  needed.  The  current  version  of  Kestrel  is  being  used  extensively  in  the 
government  aircraft  acquisition  process,  and  is  being  evaluated  by  industry 
for  suitability  to  aircraft  acquisition.  Current  capability  is  demonstrated  for 
a  maneuvering  aircraft  coupled  with  control  surface  movements,  aeroelastic 
effects,  store  separation,  and  cargo  deployment.  In  addition,  several  future 
capabilities  currently  in  prototype  are  discussed. 


I.  Introduction 

'  I  ’HE  Department  of  Defense  High  Performance  Computing  Modernization  Program  (DOD  HPCMP) 
^  has  been  involved  in  an  initiative  to  improve  DOD  acquisition  program  timeline,  cost,  and 
performance  through  the  use  of  Computational  Science  and  Engineering  (CSE)  tools  for  ships,  aircraft, 
and  antenna  design  and  analysis.  The  proposed  method  of  improving  acquisition  is  by  a  paradigm  shift 
from  “design-build-fly-fix”  to  “design-simulate-fix-build-fly.”  It  is  well  known  that  the  majority  of  life 
cycle  costs  are  established  early  in  the  design  process  when  the  least  amount  of  information  is  known 
about  deficiencies  in  the  design.  This  is  true  because  deficiencies  are  discovered  primarily  in  flight  test 
when  they  are  the  costliest  to  fix.  The  new  methodology  uses  high  resolution,  multi-disciplinary, 
simulations  to  allow  discovery  of  these  deficiencies  early  in  the  design  phase  before  the  majority  of  costs 
are  fixed.  Fulfillment  of  this  paradigm  shift  requires  system-level  high-resolution  simulation  capabilities 
that  did  not  exist  at  the  time  the  DoD  initiative  started  and  is  the  focus  of  the  resulting  program. 

The  resulting  effort  is  called  the  Computational  Research  and  Engineering  Acquisition  Tools  and 
Environments  (CREATE)  Program  [1]  established  in  October  of  2007.  HPCMP  CREATE  is  a  tri-Service 
(Army/Navy/Air  Force)  program  with  10  software  products.  The  air  vehicle  portion  of  CREATE  is 
referred  to  as  CREATE™-AV  [2]  and  has  three  software  products  CREATE™-AV  Kestrel  [3-11],  a  high 
fidelity  fixed  wing  vehicle  simulation  tool,  CREATE™-AV  Helios  [12-14],  a  high  fidelity  rotary  wing 
vehicle  simulation  tool,  and  CREATE™-AV  DaVinci  [15-16],  a  conceptual  through  detail  aircraft  design 
tool.  In  addition  to  these  three  products,  an  emphasis  on  propulsion  systems  and  their  integration  into  air 
vehicles  is  incoiporated  into  each  of  the  three  products,  referred  to  as  CREATE™-AV  Firebolt  [17-20]. 

There  are  several  over-arching  design  philosophies  being  applied  to  CREATE  software  products  [1]. 
The  first  is  scalability  on  next-generation  computer  architectures  with  linear  scalability  targets  for  Kestrel 
on  the  order  of  104  cores.  Another  CREATE  design  philosophy  is  a  “legacy  to  native”  software 
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development  approach  with  near-term  legacy  software  being  rewritten  or  factored  in  to  native  software 
over  the  life  of  the  program.  The  current  version  of  Kestrel  (version  5)  represents  a  mix  of  native 
components  with  some  legacy  libraries  (e.g.  parMETlS  [21],  SAMRAI  [22],  libmo  [3]).  One  exception  to 
this  is  the  finite-element  structural  component  used  in  collaboration  with  the  Department  of  Energy’s 
(DOE)  Sandia  National  Laboratories,  called  Sierra/SD  (formerly  referred  to  as  Salinas)  [23,24]. 
Sierra/SD  will  continue  on  its  development  path  with  the  DOE  and  be  incorporated  as  a  whole. 

Another  important  CREATE  software  design  philosophy  is  modularity.  A  common  architecture  in 
CREATE™-AV  is  a  Python-based  infrastructure  and  executive,  and  either  Python,  C,  C++,  or  Fortran 
components.  This  level  of  modularity  allows  a  build-up  approach  to  adding  capability  and  multi¬ 
disciplinary  physics.  The  resulting  factored  approach  to  the  software  aids  in  code  maintenance  and 
supportability.  A  more  detailed  look  at  the  Kestrel  architecture  is  presented  below. 

One  of  the  most  important  CREATE  design  philosophies  is  to  follow  a  professional  software 
development  process  incorporating  configuration  management,  automatic  unit,  integration,  and  system 
testing,  and  user  support.  To  have  the  desired  impact  on  the  DOD  acquisition  processes,  the  software  has 
to  be  maintainable  through  the  life  of  the  program.  All  CREATE™-AV  products  have  strict  version 
control  and  configuration  management  through  a  Subversion  (SVN)  repository  and  Kestrel  has 
continuous  integration  through  nightly  unit,  integration,  and  system  testing.  Kestrel  v5  currently  has  over 
14,000  unit,  integration,  and  system  tests  all  integrated  into  an  overnight  regression  test  harness.  User 
support  has  both  issue  tracking  and  multi-layer  live  customer  support. 

The  Kestrel  software  product  is  a  modularized,  multi-disciplinary  fixed-wing  virtual  aircraft 
simulation  tool  incoiporating  aerodynamics,  structural  dynamics,  kinematics,  and  kinetics.  Kestrel  is 
targeted  to  subsonic,  transonic,  and  supersonic  flight  conditions.  The  following  sections  present  the 
Kestrel  software  architecture,  as  well  as,  Kestrel  current  capabilities,  and  future  capabilities. 

II.  Kestrel  Software  Architecture 

The  Kestrel  architecture  is  a  blend  of  the  CREATE  design  philosophies  discussed  in  the  previous  section. 
There  are  many  levels  of  modularity  possible  within  the  architecture  but  Kestrel  implements  a  fairly  low- 
level  component  design.  This  choice  of  modularity  allows  a  component  such  as  the  computational  fluid 
dynamics  (CFD)  solver  to  be  rewritten  quickly  as  methodologies  mature,  because  the  component’s  only 
purpose  is  to  bring  the  solution  from  one  time  level  to  the  next.  Auxiliary  functions,  such  as  mesh 
management  or  output,  are  handled  in  other  components.  It  is  a  modular  approach,  factoring  traditional 
monolithic  solvers  into  the  CREATE™-AV  Common  Scalable  Infrastructure  (CS1)  piece;  components  to 
perform  fluid  dynamics,  structures,  kinematics  and  kinetics  and  other  analysis;  and  the  Kestrel  User 
Interface  (KU1).  Figure  1  depicts  a  notional  view  of  the  Kestrel  software  architecture.  The  CS1  is  an 
event-driven  Python  infrastructure  that  is  component-unaware.  The  components  themselves  can  produce 
or  respond  to  events  and  subscribe  to  or  publish  data. 
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Figure  1 :  Kestrel  Architectural  Design. 


This  allows  the  infrastructure  and  executive  to  be  coded  once  and  the  Extensible  Markup  Language 
(XML)  input  file  to  specify  the  use-case  and  contributing  components.  The  inputs  to  CS1  are  read  in  from 
an  XML  file  generated  by  the  KU1.  Efficient  data  handling  by  CS1  is  accomplished  by  passing  pointers  to 
“heavy-weight”  data  or  scalars.  The  resulting  overhead  was  measured  at  less  than  1%  compared  to  a 
monolithic  solver.  A  detailed  look  at  the  Kestrel  CS1,  KUI,  and  the  components  is  found  in  reference  [4]. 

In  Figure  1  there  are  two  dashed  boxes  surrounding  the  components.  The  left-hand  box  denotes  those 
components  that  are  shared  objects  within  the  CS1  and  maintained  by  the  Kestrel  development  team.  The 
right-hand  box  represents  executables  or  shared  objects  from  outside  sources  that  will  exchange 
information  via  the  Data  Warehouse  in  the  CS1  and  the  established  set  of  shared  data  items.  This  feature, 
used  by  a  few  collaborators  and  referred  to  as  the  Kestrel  Software  Developer  Kit  (SDK),  is  discussed  in 
reference  [4],  and  is  intended  to  allow  industry  or  commodity  software  to  work  with  Kestrel  without 
significant  rewrites  of  their  software.  An  example  use  of  this  feature  is  to  allow  a  commodity  CFD  solver 
to  be  used  with  all  of  the  other  components  in  Kestrel.  Another  example  use  woidd  be  to  incorporate  a 
“black-box”  automatic  flight  control  system  from  another  contractor  into  the  simulation. 

There  are  two  new  capabilities  in  Kestrel  v5  that  also  bring  new  components  with  them.  The  first  is 
an  improved  fluid-structure  interaction  (FS1)  capability  that  adds  a  new  finite-element  structural  solver 
component  to  the  currently  available  modal  structural  solver  component.  Kestrel  v5  expands  the  ability  to 
perform  aeroelastic  simulations  by  coupling  the  flow  solver,  KCFD,  with  a  structural  dynamics  finite- 
element  analysis  package,  Sierra/SD  [23,24],  developed  by  DOE’s  Sandia  National  Laboratories.  The 
forces  and  displacements  are  interpolated  between  the  two  models  (KCFD  and  Sierra/SD)  using  the  same 
spline  interpolation  schemes  as  are  used  with  the  modal  structural  solver,  but  the  information  exchange 
only  occurs  once  each  iteration,  reducing  the  accuracy  of  the  coupled  simulation  to  lst-order.  Future 
improvements  to  the  Sierra/SD  component  will  allow  coupling  that  results  in  a  scheme  with  globally  2nd- 
order  temporal  accuracy.  The  Sierra/SD  integration  is  discussed  in  more  detail  in  reference  [7]. 

The  other  new  capability  is  improved  aerodynamics  through  a  hybrid  unstructured  near-body  solver 
coupled  to  a  higher-order  Cartesian  off-body  solver  that  incorporates  in-line  adaptive  mesh  refinement 
and  load  balancing.  This  hybrid  approach  has  been  used  extensively  in  Helios  to  simulate  rotor  wake- 
fuselage  interactions  [13,14].  The  near-body  solver  is  the  current  KCFD  component  unchanged.  The  off- 
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body  Cartesian  solver  is  an  enhancement  to  the  SAMCart  component  used  in  the  Helios  rotorcraft 
product.  The  original  Helios  implementation  focused  on  the  rotorcraft  flowfield  requirements  and  was 
called  SAMARC  [13,14],  Eymann  et.  al.  [6]  enhanced  the  original  component  to  include  implicit  time¬ 
stepping,  shock  capturing  for  transonic  and  supersonic  fixed-wing  applications,  and  improved  turbulence 
modeling.  The  enhanced  SAMCart  component  is  currently  being  used  in  Kestrel  and  will  be  used  in 
Helios  future  versions.  A  detailed  look  at  how  SAMCart  uses  the  SAMRAI  [22]  automatic  mesh 
generation  and  load  balancing  routines,  as  well  as  a  discussion  of  the  modified  Cartesian  flow  solver  can 
be  found  in  reference  [6].  The  near-body  and  off-body  solvers  couple  at  the  subiterative  level  using 
PUNDIT  [12],  an  implicit  hole-cutting  domain  connectivity  component  to  calculate  donor  and  receptor 
cells,  as  well  as  interpolation  weights  for  data  transfer  between  the  respective  meshes,  previously  used  in 
Kestrel  for  unstructured-to-unstructured  mesh  overset  connectivity. 

III.  Kestrel  Current  Capability 

The  development  schedule  of  Kestrel  is  broken  up  into  two-week  iterations  following  an  agile 
software  development  process.  These  two-week  iterations  result  in  incremental  tested  and  documented 
capability  that  is  available  to  the  user  long  before  the  targeted  version  is  complete.  Iteration  flexibility 
also  allows  the  Quality  Assurance  and  Development  Teams  to  work  together  to  satisfy  user  issues  and 
high-priority  interrupt  requirements.  Below  is  a  bulleted  list  of  Kestrel  v5  capabilities. 

Kestrel  v5  Capability: 

1.  Static  Rigid  Aircraft 

•  Single-  and  Multi-Body 

2.  Prescribed  and  Predicted  6DoF  Motion  of  Rigid  Aircraft 

•  Single-  and  Multi-Body 

•  Prescribed  Control  Surfaces 

3.  Static,  Prescribed,  and  Predicted  Aeroelastic  Single-Body  Aircraft 

•  Modal  Structural  Coupling 

•  Finite-Element  Structural  Coupling 

4.  Aircraft-Engine  Integration  Model  Inclusion  in  1-3  Above 

•  Propulsion  Cycle  Analysis 
Fully-Coupled 

5.  All  Unstructured  Mesh  or  Hybrid  Near-Body/Off-Body  Solution  Paradigm 

•  Homogeneous  Unstructured  Mesh  Overset 

•  Hybrid  Unstructured/Cartesian  Mesh  Overset 

Kestrel  major  versions  are  released  annually.  Kestrel  v5.3  is  currently  in  general  release  to  users. 
Kestrel  v6  is  currently  in  Quality  Assurance  (QA)  testing,  and  Kestrel  v7  is  in  design  and  implementation. 

IV.  Kestrel  Version  5  Sample  Case 

This  section  presents  a  sample  case  of  a  multi-disciplinary  solution  computed  with  Kestrel  v5.  This 
sample  case  is  one  of  four  different  multi-disciplinary  cases  described  in  detail  in  [5],  These  cases  all  have 
combinations  of  traditional  capabilities  in  use  at  the  same  time  (e.g.,  maneuvers  with  either  control 
surfaces  moving,  or  aeroelastic  bodies,  or  store  separation,  as  well  as  six  degree-of-freedom  cargo 
deployments). 

Sample  Complex  Case  -  Maneuvering  F-16  Releasing  Multiple  Stores 

The  sample  case  involves  an  F-16  multiple  store  release  while  in  a  Wind  Up  Turn  (WUT)  maneuver. 
The  WUT  motion  is  initiated  by  accelerating  the  aircraft  from  a  Mach  number  of  zero  to  a  Mach  number 
of  0.6.  After  stabilizing  at  this  Mach  number  for  1  sec,  over  the  next  few  seconds  the  aircraft  rolls  to  a 
bank  angle  of  greater  than  90  degrees.  For  the  remaining  6  seconds,  the  hank  angle  is  relaxed  to  less  than 
80  degrees,  and  the  pitch  angle  is  changed  to  establish  the  maximum  angle-of-attack.  The  left  store  is 
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released  at  t=5.5  seconds  and  the  right  store  is  released  at  t=6.0  seconds.  The  store  used  is  a  500  lb  class 
GBU-38  guided  munition. 

The  aerodynamic  volume  mesh  for  the  F-16  used  in  the  simulation  has  6.5M  tetrahedra,  3.8M  prisms, 
and  19. 5K  pyramids.  The  aerodynamic  volume  mesh  used  for  the  GBU-38  in  the  simulation  has  1.6M 
tetrahedra,  2.4M  prisms,  and  14.4K  pyramids.  The  same  GBU-38  mesh  is  used  for  both  the  left  and  right 
stores.  The  combined  system  of  three  meshes  is  depicted  in  Figure  2.  The  three  unstructured  mesh  system 
was  solved  with  the  unstructured  to  unstructured  overset  capability  in  Kestrel. 


Figure  2:  Surface  mesh  of  an  F-16  with  two  under-wing  GBU-38  stores  placed  approximately  at 

the  mid-wing  pylon  stations. 

The  simulation  was  performed  using  a  time-step  of  0.0005  seconds,  3  subiterations,  and  inner  sweeps 
that  converge  the  linear  problem  to  a  residual  of  1.0x1  O'6.  Figure  3  is  a  series  of  snapshots  of  the 
maneuver  and  store  releases  at  various  points  in  time,  depicting  surface  pressures  and  iso-surfaces  of 
vorticity  magnitude  of  100/sec.  Figure  3a  depicts  the  F-16  with  the  left  and  right  store  in  carriage  position 
at  t=2  seconds  when  the  aircraft  is  beginning  the  roll  to  the  right.  Figure  3b  depicts  the  F-16  with  both 
stores  in  carriage  at  t=4  seconds,  after  the  bank  has  been  established  but  before  the  aircraft  loads  up  to 
maximum  angle-of-attack.  Figure  3c  depicts  the  aircraft  at  t=6  seconds,  0.5  seconds  after  the  left  store 
was  released  and  at  the  instant  when  the  right  store  was  released.  Figure  3d  depicts  the  aircraft  at  t=6.5 
seconds  where  it  is  clear  that  both  stores  have  been  released,  and  they  are  traveling  along  a  nearly 
horizontal  trajectory  away  from  the  aircraft  as  it  continues  the  WUT.  Finally,  Figure  3e  depicts  the 
aircraft  and  stores  at  the  end  of  the  10  second  simulation.  It  is  clear  that  the  stores  have  traveled 
significantly  away  from  the  aircraft,  but  are  still  flying  primarily  at  the  same  altitude  as  the  F-16.  At  this 
point  the  store  meshes  are  interpolating  data  from  a  very  coarse  part  of  the  aircraft  mesh,  and  the  fluid 
solution  is  somewhat  non-physical  though  stable.  Flowever,  the  inertial  motion  of  the  stores  dominates 
the  trajectory  at  this  point,  and  the  flow  solution  accuracy  is  not  critical. 

Use  of  Kestrel’s  Near  Body-Off  Body  mesh  paradigm  would  alleviate  these  inaccuracies  and  will  be 
used  for  future  store  release  cases.  Analyzing  the  positions  of  the  three  bodies  as  a  function  of  time  shows 
the  stores  traveling  down  range  for  over  1.26  miles  with  a  vertical  drop  of  833  feet  of  altitude  during  the 
full  10  seconds.  A  simulation  of  this  complexity  and  accuracy  can  aid  in  developing  new  methods  of 
weapons  delivery.  For  example,  it  is  not  typical  for  DOD  aircraft  to  release  stores  during  a  loaded  turn, 
but  if  this  became  desirable  for  some  operational  need  in  the  future,  Kestrel  could  simulate  it. 
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Figure  3:  Snapshots  of  surface  pressures  and  iso-surfaces  of  vorticity  magnitude  equal  to  100/sec 
for  an  F-16  with  left  and  right  under-wing  stores  releasing  at  5.5  and  6.0  sec,  respectively,  for  a 
Mach  number  of  0.6.  The  maneuver  is  a  10  sec  maximum  angle-of- attack  wind-up  turn.  Images  are 
at  various  time  steps,  a)  t=2.0  sec,  b)  t=4.0  sec,  c)  t=6.0  sec,  d)  t=6.5  sec,  e)  t=10  sec. 


V.  Kestrel  Future  Direction 

There  are  several  areas  of  active  Kestrel  development  in  the  Kestrel  Fixed-Wing  and  Propulsion 
Groups,  as  well  as  the  CREATE™-AV  Quality  Assurance  Team.  Kestrel  v6  includes  cross-cutting 
improvements  in  speed,  scalability,  and  accuracy,  and  a  significant  reduction  in  required  memory  per  core 
for  simulations  with  a  large  number  of  surface  elements. 

One  of  the  much-anticipated  Kestrel  v6  capabilities  is  the  Propulsion  Capability  2  that  introduces 
rotating  turbo-machinery  for  simulation  of  complex  compressors  integrated  into  the  inlet.  Nichols  et  al. 
[18]  present  results  for  turbo-machinery  component  cases  of  increasing  complexity,  including  the 
Rotor35,  Stage35,  Rotor67,  and  Rotor67  in  a  serpentine  inlet  duct  comparison  cases.  Klepper  et  al.  [19] 
also  present  simulations  of  the  full  A- 10  aircraft  with  one  of  the  engines  simulated  with  engine  cycle 
analysis  and  the  other  engine  simulated  with  an  installed  compressor  fan  rotating  at  6500  RPM. 

Another  new  capability  coming  in  Kestrel  v6  related  to  propulsion  integration  is  the  capability  to 
simulate  the  exit  nozzle  variable  throat  and  exit  plane.  Masters  [20]  presents  results  for  a  variable 
geometry  nozzle  for  the  F-110  engine  installed  in  the  F-16XL.  The  variable  nozzle  geometry  can 
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currently  be  prescribed  as  a  function  of  time,  but  will  eventually  be  tied  to  the  propulsion  cycle  analysis 
model  of  the  engine  to  automatically  drive  the  geometry  variation  with  throttle  position  and  flight 
condition. 

The  final  new  capability  to  be  discussed  is  the  ability  to  couple  an  automatic  flight  control  to  Kestrel 
to  drive  the  motion  of  bodies,  or  control  surfaces,  or  modify  flowfield  source  terms  in  response  to  the 
flowfield.  Forsythe  et.al.  [10]  present  simulations  of  a  helicopter,  with  a  modeled  actuator  disk  driven  by 
the  CASTLE  [10]  aircraft  simulation  tool,  landing  on  a  carrier  deck.  This  work  demonstrates  the  ability  to 
couple  a  “black-box”  external  executable  to  Kestrel  using  memory  on  the  cores,  as  opposed  to  a  file  10 
approach  (FIFO)  by  using  the  Kestrel  Software  Developer  Kit  [4], 

VI.  Conclusions 

In  conclusion,  significant  progress  has  been  made  since  the  inception  of  the  CREATE  Program  in 
October  of  2008.  In  the  CREATE™-AV  Kestrel  Team,  requirements  have  been  gathered,  refined,  and 
prioritized;  software  capabilities  to  meet  the  requirements  have  been  defined;  the  software  has  been 
designed,  implemented,  tested,  and  distributed  to  users  for  version  1  through  5;  version  6  is  in  alpha 
testing;  and  version  7  is  in  design.  The  Kestrel  software  product  has  been  developed  following  the  over¬ 
arching  design  philosophies  of  the  CREATE  Program  and  to  meet  the  specific  needs  of  the  aircraft 
acquisition  engineer  and  scientist  communities.  In  addition,  Kestrel  has  been  developed  from  the  ground 
up  following  sound  software  development  practices,  and  innovative  solutions  to  the  user  requirements 
have  resulted  from  the  process.  An  intensive  validation  and  verification  effort  is  perpetually  underway  to 
ensure  Kestrel  is  suitable  for  production  use  by  the  acquisition  community.  A  sample  result  for  the  v5 
release  capabilities  has  been  presented  and  a  look  into  capabilities  currently  in  prototype  that  will  be 
available  in  the  near  future  have  also  been  presented. 
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